Problem  representation  and  variation  in  the  forecasts  of  ‘political  experts’ 


Helen  E.  Purkitt 
Dept,  of  Political  Science 
US  Naval  Academy 
Annapolis,  MD  21402 
Email:  purkitt@nadn.navy.mil 

ABSTRACT 

Forecasts  often  involve  judgments  about  “ill- structured”  problems  that  lack  an  optimal 
solution  or  agreement  about  boundary  conditions.  This  paper  describes  how  the  degree  of 
structure  of  the  problem  and  the  forecasting  time  frame  can  be  used  to  form  a  classification 
scheme  of  different  types  of  forecasting  problems.  This  scheme  can  be  used  as  a  guide  for 
choosing  the  most  appropriate  approach  and  methods  to  use  in  analyses  and  predictions.  The 
paper  next  describes  a  method  for  summarizing  how  individuals  or  groups  of  problem  solvers 
analyze  “ill- structured”  social  and  political  problems.  Examples  of  this  method,  based  on  the 
construction  of  free-hand  causal  diagrams  of  novice  and  expert  political  forecasters,  are 
presented.  Experience  to  date  using  these  schematic  summaries  indicate  that  such  summaries  are 
useful  for  understanding  the  reasoning  processes  of  clients  and  to  check  the  reasoning  process 
used  by  experts  from  different  backgrounds  and  experiences  in  making  forecasts. 
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Introduction 

Many  political  forecasting  exercises  focus  on  discrete  events  such  as  election  outcomes.  Other 
forecasting  efforts  in  American  Politics  examine  similarly  “well-structured”  policy  outcomes 
such  as  governmental  appointments  or  future  budgets  (Sigelman,  Batchelor,  Stekler,  1999: 

125).  A  similar  pattern  is  evident  in  forecasting  research  in  International  Relations.  In  this  field, 
much  of  the  forecasting  research  has  tried  to  develop  aggregate  models  to  predict  discrete  policy 
events  such  as  political  coups  (Johnson,  Slater  and  McGowan,  1984)  or  the  outbreak  of  war 
(Singer  and  Small,  1968).  In  contrast,  government  officials  and  corporate  clients  are  usually 
interested  in  political  risk  forecasts  of  complex  and  highly  uncertain  problems.  Modeling  efforts 
often  trigger  disagreements  about  underlying  assumptions  or  relationships  among  variables 
because  there  is  no  general  agreement  on  how  to  define  the  problem  at  hand.  A  lack  of 
agreement  on  fundamental  aspects  of  the  problem  is  also  fueled  by  high  levels  of  uncertainty 
about  probability  outcomes  or  scenarios. 

The  pervasiveness  and  importance  of  these  poorly  structured  and  uncertain  problems  in 
the  political  realm  has  led  some  researchers  interested  in  socio-political  forecasts  to  use  insights 
generated  by  research  on  how  people  solve  a  variety  of  problems  (Voss,  1998).  A  recurring 
finding  from  this  diverse  body  of  research  is  that  it  is  critical  to  understand  how  individuals  and 
groups  develop  a  mental  model  or  Ashared  representation  of  problem©  to  understand  subsequent 
decision  making,  choice  and  forecast  (Sylvan  and  Voss,  1998). 
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This  paper  builds  upon  research  using  the  concept  of  “shared  problem  representation”  to 
understand  better  how  political  risk  forecasts  of  Aill-structured@  problems  are  reached.  Ill- 
structured  problems  are  a  class  of  problems  that  lack  a  single  or  optimal  solution.  The  problems 
are  often  poorly  structured  because  there  is  a  lack  of  agreement  about  the  assumptions  and 
implicit  model  s  that  shape  how  novice  or  expert  forecasters  frame  the  problem.  This  paper  uses 
the  idea  of  problems  having  varying  degrees  of  structure  to  develop  a  classification  scheme  of 
different  types  of  problems.  The  classification  scheme  is  based  on  two  dimensions:  1)  the  degree 
of  structure  of  the  problem  at  hand  and,  2)  the  time  perspective  of  the  prediction,  i.e.,  whether  it 
is  a  short-term,  medium-,  or  long-term  forecast.  This  scheme  is  offered  as  one  approach  for 
obtaining  a  better  understanding  of  the  best  forecasting  approach  and  methods  to  use  in  making 
predictions.  Underlying  the  scheme  is  the  assumption  that  if  end  users  of  systematic  predictions 
understand  and  agree  with  the  way  the  prediction  problem  is  define,  they  will  be  more  likely  to 
accept  and  use  the  obtained  results. 

Many  other  factors  may  effect  whether  the  results  of  forecasting  efforts  of  ill- structured 
problems  are  actually  used.  Two  factors  stand  out  as  especially  difficult  ones  for  political 
forecasting  efforts.  One  is  cognitive  conceit  or  the  pervasive  tendencies  of  people  to  have  an 
unwarranted  faith  in  the  adequacy  of  their  intuitive  analyses.  A  second  one  relates  to  the  fact  that 
political  expertise  is  often  tied  to  one=s  position  or  reputation  as  an  expert  rather  than  mastery  of 
a  set  of  problem  solving  skills.  These  two  factors  often  are  critical  in  determining  whether  the 
results  of  systematic  forecasting  efforts  will  in  fact  be  used.  The  paper  concludes  with  a 
description  of  one  method  that  can  be  used  to  explicate  the  usually  implicit  models  of 
forecasters,  analysts,  or  end  users.  The  method  is  based  on  free-hand  causal  schematic 
summaries  of  either  the  think  aloud  protocols  of  individuals  or  the  shared  problem  representation 
developed  by  a  small  group.  Two  examples  of  problem  representation  diagrams  are  presented  to 
illustrate  how  the  method  can  be  used  to  better  understand  the  reasoning  processes  and 
performance  accuracy  of  political  clients  or  political  Aexperts@  who  have  very  different 
background  and  experiences.  Use  of  this  approach  may  also  help  in  communicating  and  gaining 
client=s  acceptance  of  the  results  of  complex  forecasting  tasks. 

Classifying  forecasting  problems  based  on  the  degree  of 
problem  structure  and  the  time  frame  of  the  prediction 

Table  1  is  a  classification  scheme  that  is  based  on  two  fundamental  dimensions  of  social, 
economic  or  political  problems.  The  types  of  problems  studied  by  social  scientists  are  classified 
by  the  values  of  two  variables:  the  degree  of  structure  of  the  problem  to  be  analyzed  and  the  time 
frame  of  the  prediction. 

The  degree  of  problem  structure  refers  to  the  extent  that  there  is  an  inter- subjective 
consensus  among  experts  on  the  boundary  conditions,  the  parameters  and  problem  solving  logics 
that  should  be  used  to  obtain  a  correct  or  optimal  solution  for  a  particular  type  of  problem.  Well- 
structured  problems  are  ones  that  have  a  high  degree  of  agreement  about  what  constitute  a 
correct  or  optimal  solution.  At  the  other  end  of  this  continuum  are  Aill-structured  problems @ 
where  there  is  little  or  no  agreement  about  how  to  define  or  represent  the  problem.  Variation  in 
problem  structure  can  also  refer  to  the  extent  to  which  there  is  agreement  on  the  important 
variables,  the  roles  or  relationships  among  these  variables  within  a  model  or  the  specificity  with 
which  variables  are  conceptualized  and  measured. 

This  continuum  is  based  on  past  problem  solving  research  that  suggests  that  people  use 
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different  types  of  problem  solving  logics  to  solve  different  types  of  problems  (Simon,  1973; 

Voss,  1998;  Voss,  Greene,  Post  and  Penner,  1983).  The  basic  thesis  of  this  paper  is  that  forecast 
methods  and  forecasts  will  be  more  likely  to  be  accepted  and  used,  if  the  results  are  related  to  the 
implicit  model  of  the  problem  employed  by  end  users.  Psychologists  who  study  problem-solving 
behavior  make  a  distinction  between  Awell- structured @  and  Aill- structured @  problems  (Voss, 
1998).  Highly- structured  problems  have  well-known  boundary  conditions,  generally  agreed-upon 
or  optimal  solutions,  and  little  or  no  uncertainty  associated  with  the  correct  or  known  solution.  In 
contrast,  ill-structured  problems  are  more  ambiguous  types  of  problems  with  no  agreed-upon 
definition  or  boundary  conditions.  There  is  usually  a  large  degree  of  uncertainty  associated  with 
the  outcomes  of  ill- structured  problems  because  this  class  of  problems  has  no  single  optimal 
solution  or  often  no  agreement  on  the  correct  solution  paths. 

The  relevant  time  frame  for  a  prediction  is  a  second  fundamental  dimension  of  any 
forecast.  Past  social  and  cognitive  research  suggests  that  when  people  make  decisions  or 
forecasts  intuitively  they  typically  employ  vague  distinctions  such  as  tomorrow,  the  next  couple 
of  weeks,  and  an  ill-defined  longer  term  future.  This  longer  term  can  range  from  next  year  to  an 
unspecified  future.  The  width  of  class  intervals  and  the  concepts  used  to  distinguish  between  the 
short-term,  medium-term  and  long-term  seems  to  vary  with  the  type  of  problem  being  examined. 
Thus,  the  desire  for  certainty  and  demands  for  specificity  are  undoubtedly  greater  when  the 
forecast  is  associated  with  a  diagnosis  of  a  life  threatening  disease  than  when  the  forecasting  task 
is  one  of  deciding  which  class  intervals,  operational  indicators  and  data  should  be  use  to  make 
the  most  accurate  age-groups  projections  about  average  life  expectancy. 

Despite  these  differences  there  is  one  important  similarity  between  the  two  types  of 
problems.  This  commonality  is  the  evidence  that  has  been  accumulating  for  decades  that  humans 
generally  are  only  capable  of  using  a  limited  number  (e.g.,  2-5)  of  analytical  dimensions 
systematically  in  their  intuitive  analyses  and  predictions.  Thus,  limitations  in  the  capacity  of 
working  memory  ensure  that  there  will  be  recurring  patterns  in  the  time  frame  used  in  intuitive 
analysis  of  different  problems  within  a  particular  problem  area  or  domain.  For  example  in  pubic 
policy  analyses,  the  short-term  for  most  policy  makers  is  typically  a  few  days  or  weeks.  The 
medium  term  typically  refers  to  several  weeks  or  month,  and  the  long-term  is  usually  a  vague 
time  reference  beyond  the  next  budgetary  or  election  cycle. 

By  segmenting  these  two  dimensions  into  three  categories,  it  is  possible  to  illustrate  how 
this  classification  system  can  be  used  to  distinguish  different  types  of  problems  in  a  particular 
field.  This  table  is  designed  to  be  illustrative  of  the  different  types  of  problems  that  analysts  and 
practitioners  often  encounter  in  the  course  of  making  political  forecasts  about  future  domestic 
and  international  politics. 
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Table  1 


SPECIFICITY  OF  PROBLEM 


Well-specified  Moderately  Ill-structured 

problems  structured  problems 

problems 


T 

I 

M  Short- 

E  term 

Will  the  President’s 
popularity  in  the  polls 
be  higher  or  lower 
tomorrow? 

Will  the  President 

renounce  the  ABM 
Treaty  tomorrow? 

What  will 

Saddam 

Hussein  do 

tomorrow? 

Medium 

term 

Will  Republicans  gain 
control  of  the  Senate  in 
the  2002  elections? 

Will  the  Feds  lower 
interest  rates  again  in 
2001? 

Which 

Presidential 

policy 

initiative  will 
be  judged  the 
most 

successful  one 
initiated 
during  2001/2? 

F 

R  Long 

A  term 

M 

E 

How  many  new  eligible 
votes  will  actually  vote  in 
2004? 

Will  there  be  a 
coup  in  Nigeria  during 
next  5  years 

Which  states 
will  Afail@in 
the  future 

The  first  row  of  the  table  provides  examples  of  short-term  political  forecasting  questions 
of  varying  degrees  of  problem  specificity.  The  first  two  cells  are  examples  of  a  well- specified 
forecasts  that  can  be  estimated  using  simple  extrapolation  logic.  The  third  example  illustrates  the 
type  of  short-term,  ill- structured  problem  that  senior  government  officials  in  all  countries  want 
their  intelligence  service  to  answer  with  complete  certainty.  Even  though  the  dependent  variable 
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or  prediction  to  be  made  is  not  specified,  policy  makers  want  analyses  and  predictions  to  cover 
all  likely  contingencies.  This  demand  is  even  greater  when  the  subject  is  an  actor  such  as 
Saddam  Hussein  who  has  proven  to  be  very  adept  at  using  surprise  moves  to  his  advantage. 

The  medium  term  prediction  questions  also  involve  problems  with  varying  degrees  of 
specificity,  uncertainty  about  the  most  probable  outcome,  or  the  meaning  of  the  phenomena  to  be 
predicted.  The  first  example  taps  a  type  of  political  problem  that  can  be  modeled  using  a 
regression  equation  and  only  a  few  independent  variables  (  see  election  analyses  in  Sideman, 
Batchelor,  and  Stekler,  1999).  These  models  can  be  used  to  predict  election  outcomes  as  long  as 
the  end  user  is  willing  to  accept  the  caveat  of  Aall  other  conditions©  remaining  constant. 
However,  predictions  would  have  to  be  conditional  and  updated  since  the  values  of  key 
variables  like  candidate  image,  what  campaign  issue(s)  will  be  identified  as  the  most  important 
ones,  and  voter  turn-out  rates  among  undecided  and  weak  partisans  will  change  between  now 
and  the  next  election. 

The  second  question  is  moderately  well- structured  and  has  also  been  modeled  extensively 
using  economic  models  based  on  free-market  principles  of  supply  and  demand  of  several  key 
parameters.  However,  there  is  a  some  uncertainty  in  generated  predictions  tied  to  the  fact  that 
decisions  made  by  the  members  of  the  Federal  Reserve  Board  often  take  into  account  other 
information  and  values  than  those  incorporated  in  economic  models.  Moreover,  the  supply-and- 
demand  free-market  model  often  generates  predictions  that  are  incorrect  in  the  short-to-medium 
term.  What  is  interesting  about  this  type  of  forecast  is  that  so  many  people  continue  to  believe  in 
the  validity  of  the  economic  model  even  though  many  predictions  (e.g.,  about  the  rise  of 
inflation  associated  with  sustained  economic  growth)  have  not  been  accurate  for  near  term  trends 
in  the  United  States. 

Ill- structured  problems  that  ask  for  medium  term  predictions  often  approximate  the  one 
listed  in  the  last  cell  of  row  2.  This  type  of  problem  is  difficult  to  predict  in  part  because  the 
implications  of  the  party  leadership  change  in  the  US  Senate  on  Bush’s  legislative  agenda  is  not 
yet  known.  A  more  general  problem  is  that  there  is  no  objective  criteria  or  inter- subjective 
consensus  on  what  will  constitute  Asuccessful  policy  indicatives©  among  political  novices  (e.g., 
the  mass  public)  or  the  Aexperts@(e.g.,  policy  analysts  and  politicians). 

All  long-term  forecasting  problems  introduce  more  elements  of  uncertainty  due  to  the 
increased  likelihood  that  other  unspecified  factors  may  influence  the  outcome.  However,  the 
question  of  how  many  women  will  vote  in  the  2004  election  is  well-structured  and  may  be 
addressed  by  combining  extrapolations  of  recent  trends  in  such  critical  variables  as  voter 
registration  and  turn-out  by  gender.  The  second  question  can  use  existing  regression  models  that 
have  predicted  past  political  coups  with  a  high  degree  of  accuracy  (Johnson,  Slater,  and 
McGowan,  1984).  However,  even  though  this  problem  is  fairly  well- structured,  it  is  very 
complex.  Nigeria,  for  example,  may  be  a  special  case  when  compared  to  other  sub-Saharan 
African  states  because  of  it=s  size,  dependency  on  oil  revenues  for  economic  growth,  and  due  to 
a  reoccurring  pattern  of  prolonged  military  rule  following  periods  of  extreme  political  unrest  and 
economic  instability.  Since  existing  Acoup-proneness©  models  use  annual  time  series  data,  this 
data  may  be  too  crude  to  capture  the  sharp  economic  decline  and  political  unrest  that  is  likely  to 
occur  following  a  major  drop  in  oil  prices  or  the  disruption  of  oil  exports  of  this  single¬ 
commodity,  oil  exporting  economy. 1 

The  last  question  is  one  that  has  been  systematically  addressed  by  recent  modeling  efforts 
sponsored  by  the  US  government  (Esty,  et  al,  1998).  The  State  Failure  Project  has  identified  a 
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number  of  variables  that  seem  to  be  important  predictors  of  state  failure.  When  the  modeling 
effort  is  limited  to  African  states,  the  predictive  power  of  this  model  increases  significantly.  The 
State  failure  project  projections,  based  on  inductive  artificial  intelligence  classification 
techniques  rather  than  statistical  models,  has  identified  three  factors  as  the  most  efficient 
discriminations  between  Afailed  states©  and  stable  states  during  the  post-World  War  II  era:  the 
level  of  material  living  standards,  the  level  of  trade  openness;  and  the  level  of  democracy.  This 
model  had  a  70  %  success  rate  in  post-dictive  predictions  of  which  states  in  Africa  would  fail 
using  a  yes/no  dichotomous  dependent  variable.  Despite  this  reasonable  good  fit  between  data 
and  the  model,  the  acceptance  and  use  of  such  models  is  rare  among  policy  makers  One  reason 
may  be  due  to  the  fact  that  while  policy  makers  are  likely  to  want  answers  to  questions  addressed 
in  these  recent  systematic  research  efforts,  the  way  they  frame  the  question  is  likely  to  be  very 
different.  As  indicated  in  Table  1,  policy  makers  are  most  likely  to  want  specific  answer  to  a 
much  vaguer  and  more  controversial  question,  to  wit:  which  states  will  fail  and  when?  This 
question  is  difficult  to  answer  using  existing  data-based  models  because  it  is  asking  for  point 
predictions  across  an  unspecified  future  time  frame.  An  even  more  problematic  aspect  of  this 
Aill- structured©  problem  is  that  there  is  no  widely  agreed  upon  definition  of  Astate  failure.  © 

The  pervasiveness  of  Aill-structured©  problems 

Most  of  the  problems  studied  in  the  humanities  &  social  sciences  are  ill- structured 
problems.  Typically  there  is  little  or  no  agreement  on  how  to  define  or  frame  the  problem. 
Analysts  dealing  with  this  class  of  problem  rely  heavily  upon  their  prior  beliefs,  knowledge, 
experiences  and  value  preferences  to  define  the  parameters  of  a  simplified  problem 
representation.  Usually  all  relevant  information  is  not  used  to  understand  the  initial  problem 
definition  or  framing  even  though  this  initial  framing  is  critical,  a  major  determinant  of  the 
solution  or  solution  path.  This  recognition  has  led  some  researcher  to  focus  on  how  problems  that 
need  immediate  attention  are  represented.  Understanding  the  initial  framing  is  critical  since  the 
solution  or  solution  path  that  is  adopted  is  embedded  in  the  way  the  problem  is  defined. 


6 


Figure  1 


WELL-STRUCTURED  PROBLEMS 


KNOWN  OR  OPTIMAL  SOLUTION 


No  UNCERTAINTY 
correct  answer(s) 


ILL-STRUCTURED  PROBLEMS 

NO  AGREEMENT  ON  SOLUTIONS 
OR  SOLUTION  PATH 

. X 

HIGH  degree  UNCERTAINTY 
answer(s)  or  solution  paths 


Focusing  on  how  people  solve  ill- structured  problems  has  proven  to  be  a  productive 
strategy  for  identifying  recurring  patterns  in  the  way  people  think,  make  decisions,  and 
predictions  about  a  wide  range  of  problems.  In  contrast  to  the  predominant  approach  of  voting 
studies  which  focus  on  the  likely  behaviors  of  mass  public  or  voters,  many  analysts  working  at 
the  micro-level  have  found  the  concept  of  problem  representation  to  be  a  useful  device  for 
understanding  more  ambiguous  situations.  These  researchers  are  interested  in  understanding  and 
predicting  the  behavior  of  elites  and  the  processes  they  use  to  make  decisions  and  predictions. 
Typically,  the  unit  of  analysis  is  a  group  although  some  work  has  focused  on  individual  decision 
makers  or  analysts 

The  concept  of  problem  representation  has  proven  to  be  a  useful  concept  for 
understanding  how  the  options  that  were  considered  in  a  particular  situation  were  generated 
(Sylvan,  1998:3).  Cottam  and  McCoy  (1998:  116-44  )  used  the  concept  to  examine  the 
relationship  between  the  type  of  image  used  by  various  political  decision  makers=  and  the 
problem  representation  they  develop,  Gannon  (1998:  249-60  )  has  used  it  to  study  problem 
representation  from  Senate  Judiciary  Committee  hearings.  Young  (1998:  213-48)  combined  the 
concept  with  a  computational  model  to  analyze  Jimmy  Carter  speeches.  A  few  analysts  have 
used  the  concept  to  model  the  social  discourse  about  a  particular  topic.  For  example,  Rubino- 
Hallman  (1998:261-278)  used  the  concept  to  guide  the  construction  of  an  inductive  artificial 
intelligence  classification  of  the  themes  that  were  used  by  the  Presidential  Commission  on 
Women  in  Combat. 

Psychological  constraints  to  political  forecasting  of  Aill-structured@ 
political  problems:  Cognitive  conceit  and  political  expertise 

Individuals  use  simplifying  heuristics  (e.g.,  mental  decision  rules)  when  they  solve 
problems.  Individuals  and  groups  use  more  general  meta-heuristics  (e.g.,  mental  strategies)  to 
understand,  to  make  inferences  and  decisions  about  ill-structured  problems  across  a  number  of 
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different  domains  (Pious,  1993).  In  politics  one  of  the  most  pervasive  metaheuristic  is 
Asatisficing@  (Simon,  1955).  Rather  than  engage  in  comprehensive  information  search  and 
analysis  of  the  probabilities  and  utilities  associated  with  all  possible  outcomes,  Simon=s  early 
research  documented  that  people  in  public  organizations  usually  search  for  a  Agood  enough©  or 
satisfactory  solution  for  the  immediate  problem  at  hand  (Lindblom,  1959;  Wildavsky,  1964; 
Dyson  and  Purkitt,  1986). 

The  universal  reliance  on  heuristics  is  necessitated  by  the  limitations  in  the  amount  of 
information  that  can  be  processed  systematically  within  the  confines  of  working  memory.  A 
reliance  on  heuristics  leads  individuals  to  commit  a  large  number  of  cognitive  biases  or  errors  in 
judgment  when  compared  to  normative  principles  of  rational  decision  making  (Pious,  1993).  An 
extensive  body  of  experimental  research  literature  indicates  that  people  rarely  make  adequate 
adjustments  from  their  initial  anchor  or  reference  point  (Kahneman,  Slovic  and  Tversky,  1992; 
Quanttrone  and  Tversky,  1988).  Individuals  are  frequently  unaware  of  their  errors  because  there 
is  a  tendency  for  people  to  have  an  unwarranted  faith  in  the  adequacy  of  their  intuitive  judgments 
(Nisbitt  and  Ross,  1980).  This  persistence  faith  in  one=s  intuitive  analytical  processes  has  been 
called  Acognitive  conceit.© 

The  persistence  of  cognitive  conceit  helps  to  explain  why  intuitive  analysts  usually 
believe  that  they  have  used  all  relevant  information.  Nnumerous  studies,  however,  document  that 
people  have  difficulty  judging  what  information  they  actually  used  and  frequently  report  using 
more  information  than  they  did  in  experimental  tasks.  Ironically,  an  individuals  faith  in  the 
adequacy  of  prior  intuitive  predictions  seems  to  increase  with  the  amount  of  available 
information  whether  their  predictions  were  correct  or  not  (Oskamp,  1965;  Lichtenstein  and 
Fishhoff,  1977).  Overconfidence  in  the  adequacy  of  past  predictions  has  been  found  to  persist 
for  most  experimental  subjects  even  when  the  subjects  are  given  feedback  information  about  the 
inaccuracy  of  their  prior  judgments  Cognitive  conceit  seems  to  limit  the  ability  of  most 
individuals  to  recognize  inadequacies  in  their  past  intuitive  judgments  and  thus,  their  ability  to 
make  adjustments  to  the  cognitive  routines  and  models  used  to  improve  predictive  abilities.  The 
tendency  also  makes  it  difficult  for  intuitive  experts  to  correctly  calibrate  judgments  with 
incoming  information  or  to  learn  from  past  mistakes  (Pious,  1993).2 

We  see  evidence  of  the  pervasiveness  of  cognitive  conceit  among  individuals  and  groups 
involved  in  politics.  Most  policy  experts  and  policy  makers  continue  to  rely  on  their  intuitive 
analyses  and  prefer  to  use  Aseat-of-the-pants@  problem  solving  logics  even  when  they  have  the 
results  of  highly  reliable  formal  forecasting  models  available  to  them.  Tetlock  (1999)  has 
compiled  extensive  evidence  that  political  experts  and  politicians  rely  on  their  intuitive 
analytical  models  even  when  confronted  with  evidence  that  these  models  generated  incorrect 
predictions.  Even  famous  political  experts  such  as  Kissinger  and  Brezinski,  when  asked  to 
explain  a  number  of  their  more  famous  erroneous  predictions  about  future  events  and  trends  in 
foreign  policy  responded  in  very  similar  ways.  These  two  experts,  and  many  other  experts, 
usually  claim  that  they  were  Anearly  right©  (Moran,  1999). 

Cognitive  conceit  in  politics  is  compounded  by  the  fact  that  Aexpertise©  is  often 
associated  with  a  person=s  occupation,  social  position  and  reputation  as  a  political  expert.  The 
expertise  of  political  experts,  like  expertise  in  other  several  other  fields,  is  rooted  in  part  on  their 
prior  knowledge  or  cognitive  skills  at  solving  difficult  political  problems.  However,  unlike 
domains  such  as  chess  or  music  where  expertise  is  associated  with  the  mastery  of  certain 
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performance  skills,  there  is  no  generally  agreed  upon  criteria  for  distinguishing  between  political 
experts  and  novices.  Instead  political  scientists  usually  divide  a  population  into  experts  (elite, 
policy  makers  ,  or  their  advisors)  and  novices  (citizens  or  the  mass  public).  In  studies  of  the  mass 
public,  political  researchers  collect  data  on  the  extent  of  a  person=s  knowledge  of  local  or 
national  affairs,  and  whether  they  think  of  politics  in  abstract  terms  on  mainly  on  the  basis  of 
personal  experience.  Thus,  cognitive  complexity,  social  awareness  and  minimalism  provide 
standards  for  differentiating  experts  (Dawes,  1988;  Fiske  and  Taylor  1984;  Sniderman,  Brody 
and  Tetlock,  1991).  Only  a  few  studies  have  systematically  examined  the  nature  of  political 
expertise  or  the  differences  between  political  novices  and  experts. (Chan,  1982;  Dyson  and 
Purkitt,  1986;  Hermann,  Tetlock,  and  Diascro,  2001;  Taber  and  Steenbergen,  1993:  Tetlock, 
1992,  Voss,  1998). 

In  contrast,  controls  for  expert-novice  differences  is  standard  practice  in  psychology. 
Cognitive  psychologists  who  have  studied  how  experts  and  novices  solve  various  types  of 
problems  in  different  fields  have  found  some  important  differences  in  the  problem-solving 
activities  of  experts  and  novices  including:  (1)  Experts  have  a  more  extensive  and  better- 
organized  knowledge  base  than  novices;  (2)  experts  deal  more  effectively  with  problems  that 
have  structure  and  known  Abest  solutions @;  (3)  experts  tend  to  work  from  problem 
understanding  to  problem  solution  while  novices  tend  to  immediate  look  for  solutions  and  to  use 
possible  solutions  to  achieve  problem-understanding;  and  (4)  experts  draw  upon  a  larger  set  of 
heuristic  than  novices  during  problem  solving  (See  Anderson  1988;  Chi,  Glaser  and  Farr,  1988; 
Larkin  et  al.  1980;  Shanteau  1989;  and  Voss,  1998).  These  findings  help  to  explain  why  experts 
are  usually  better  prepared  to  solve  certain  types  of  problem  when  compared  to  domain  novices. 
Experts  in  many  fields  are  individuals  who  have  better  organized  memories  that  contain  more 
interrelated  cognitive  elements  and  often  have  precompiled  subroutines  to  help  them  frame  and 
then  solve  a  problem. 

Since  many  political  experts  and  politicians  gain  their  expertise  from  their  prior 
experiences  in  appointed  or  elected  positions,  the  problem  solving  behavior  of  political  experts 
will  not  necessarily  approximate  the  behaviors  typically  associated  with  expertise  in  other  fields. 
Moreover,  the  sharp  distinction  between  novices  and  experts  tends  to  blur  in  many  fields  as 
individuals  and  groups  are  asked  to  tackle  ill- structured  problems  that  cannot  be  solved  by  well 
developed  algorithm  or  problem  solving  skills.  In  politics,  where  many  problems  are  poorly 
structured  ones  without  a  known  or  single  optimal  solution,  it  is  hardly  surprising  that  the 
meaning  of  expertise  and  effective  problem  solving  skills  remains  ambiguous  and  controversial. 

A  limited  amount  of  research  on  political  expertise  has  tended  to  find  that  Aexpertise@  is 
tightly  coupled  with  verbal  articulateness  and  related  attributes  such  as  social  status  and  race 
(Chan,  1982;  Purkitt,  1998).  Researchers  have  also  found  a  great  deal  of  variation  across 
Aexperts@  in  terms  of  the  structure  and  amount  of  detail  included  in  their  cognitive 
representation  of  ill-structure  problems  (Purkitt,  1998,  Taber  and  Steenbergen,  1993). 

The  pervasiveness  of  cognitive  conceit  and  the  ambiguous  nature  of  expertise  in  politics 
are  serious  obstacles  to  those  interested  in  having  political  experts  and  policy  makers  rely  more 
on  the  results  of  formal  forecasting  methods  and  explicit  decision  aids.  Recognition  of  these 
tendencies  underscore  the  importance  of  an  analyst  having  a  thorough  understanding  of  how  the 
end  user  represents  the  forecasting  problem.  A  better  understanding  of  the  usually  implicit 
mental  representation  of  the  problem  of  the  end  user  may  help  the  forecaster  to  more  effectively 
communicate  the  results  of  forecasting  efforts. 
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Using  schematic  summaries  to  explicate  the  mental  model  of 
novice  and  expert  forecasters  &  clients 

One  approach  for  obtaining  a  better  understanding  of  how  clients  think  about  or  represent 
a  particular  forecasting  problem  is  to  ask  individuals  or  small  groups  to  Athink  aloud@  about  the 
important  features  of  the  problem.  These  verbatim  verbal  protocols  can  be  coded  using  well- 
established  protocol  coding  techniques  (Ericsson  and  Simon,  1993).  I  have  used  the  results  of 
protocol  analysis  of  two  different  sets  of  political  experts  to  determine  whether  the  individuals 
were  using  a  Alist-like@  or  Ascript-based@  approach  to  problem  representation  (1998,  2000).  I 
find  that  this  classification  exercise  helps  me  to  understand  whether  an  individuals  thought 
process  more  closely  approximated  of  experts  or  novices  in  other  fields. 

In  the  most  recent  study  (2000),  a  diverse  sample  of  64  individuals,  whose  job  involved 
working  on  environmental  and  political  problems  in  Southern  Africa,  were  asked  to  identify  the 
important  environmental  issues  or  problems  in  specific  countries,  or  in  the  Southern  African 
region  as  a  whole,  that  might  lead  to  future  political  instability  or  conflict.  Participants  in  the 
study  constituted  a  convenience  sample  of  64  individuals  working  in  a  variety  of  governmental 
and  non-governmental  organizations  in  the  United  States  and  in  six  countries  in  Southern 
Africa.3  In  this  sample,  31  of  the  participants  used  a  Alist-like@  approach  while  33  discussed 
most  issues  within  the  context  of  prior  scripts  and  beliefs.  This  dichotomy  of  problem-solving 
logic  is  not  a  logically  pure  division  as  the  problem-solving  logic  used  by  the  individuals  range 
along  a  continuum  from  simple  lists  through  a  completely  script-based  perspective  with  explicit 
assumptions,  beliefs,  inferences  and  predictions  about  future  outcomes  that  flowed  from  their 
initial  statement  of  the  problem. 

I  also  found  that  individuals  with  extensive  prior  experiences  working  in  bureaucratic 
positions  that  required  them  to  deal  with  and  to  justify  policies  related  to  a  wide  range  of 
environmental  policy  issues  (i.e.,  such  as  employees  of  the  US  DOD  Office  of  Environmental 
Security,  US  Environmental  Officers  in  foreign  embassies,  or  a  foreign  country=s  national  or 
national  security  forces  environmental  officer)  were  more  likely  to  develop  well-structured  and 
detailed  problem  representations.  In  contrast,  individuals  working  in  more  specialized  policy 
areas,  especially  those  with  little  or  no  experience  with  similar  analytical  tasks,  were  more  likely 
to  develop  a  list-like  problem  representation  that  approximates  the  general  problem-solving 
structure  of  a  Anovice.@ 

The  data  suggest  that  individuals  whose  jobs  required  them  to  analyze  the  linkages 
between  environment  problems  and  politics  used  an  existing  analytical  framework  or  political 
script  rather  than  adopt  a  list-like  approach  to  identify  environmental- security  problems.  I 
anticipated  and  found  that  individuals  working  on  specific  substantive  problem  areas  (i.e.,  water 
forestry,  agriculture)  were  more  likely  to  focus  on  problems  related  to  their  substantive  area  of 
expertise  rather  than  attempt  a  broader  regional  perspective  . 

The  majority  of  the  participants  in  this  study  focused  their  attention  primarily  on  salient 
issues  and  problems  in  one  country.  Only  twenty  of  the  64  participants  focused  on  future 
probable  political  consequences  of  current  problems  and  issues.  Of  the  20,  most  had  prior  or 
current  experience  working  in  political  positions.  Only  a  few  of  them  used  time  interval 
distinctions  when  talking  about  future  probable  events  or  trends  (e.g.,  17%  (N=ll).  Among  this 
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subset  only  a  few  made  any  effort  to  identify  when  future  probable  events  or  trends  would  be 
likely  to  occur  or  distinguish  among  short,  medium,  or  longer  term  future  trends.  The  eleven 
individuals  who  made  the  effort  were  working  on  specific  policy  problems  that  required  specific 
time  estimates  (e.g.,  future  projections  of  absolute  water  scarcities)  or  were  politicians  or 
analysts  who  were  focused  only  on  near-term  political  events  such  as  upcoming  elections. 

The  structure  of  these  problem  representations  are  consistent  with  past  information 
processing  generalizations.  Seventy- four  percent  of  the  participants  who  adopted  a  list-like 
approach  cited  five  or  more  problems  or  issues.  No  one  in  this  sub-group  listed  more  than 
twelve.  This  finding  is  consistent  with  Miller=s  (1956)  insight  that  individuals  can  only  process 
a  few  (i.e.,  7  +  or  -  2)  bits  of  information  in  working  memory.  The  individuals  who  tried  to 
discuss  specific  issues  using  a  more  general  script-based  perspective  also  quickly  experienced 
information  processing  constraints  in  efforts  to  develop  a  problem  representation.  These 
individuals  were  only  able  to  use  a  few  (i.e.,  2-5)  abstract  dimensions  in  developing  a  mental 
representation  of  the  problem.  This  result  is  also  consistent  with  a  past  research  which  has  found 
that  individuals  can  only  use  2-5  abstract  analytical  dimensions  to  systematically  analyze 
complex  problems  (see  Purkitt,  1991  for  a  review  of  this  literature  ). 

This  environmental-political  study  replicated  the  results  of  my  earlier  (1998)  study  that 
found  that  individuals  from  higher  socio-economic  status  groups  with  extensive  experience  with 
similar  verbal  tasks  tended  to  be  more  articulate  and  thus,  generated  longer  verbal  protocol  than 
non-native  English  speakers.  There  were  differences  in  the  verbal  articulateness  across  ethnic 
groups.  These  findings  are  consistent  with  other  studies  that  have  found  that  a  person=s  verbal 
articulateness  is  tightly  coupled  with  a  person=s  socio-economic  status  as  well  as  the  perception 
of  others  as  to  a  person=s  >political  expertise=  (Mohammed,  1993;  Purkitt  and  Dyson,  1988; 
Shanteau,  1989;  Voss,  Tyler,  Yengo,  1983).  I  found  that  by  summarizing  the  Athink  aloud@ 
protocols,  it  was  easier  to  identify  the  analytical  dimensions  used  by  each  respondent  in  their 
Athink  aloud@  protocol.  The  summaries  also  helped  to  disentangle  differences  in  the  analytical 
structure  and  content  of  each  participant=s  Athink  aloud@  analysis  from  Alevel  of  verbosity.  @ 
By  summarizing  verbatim  statements  associated  with  free-hand  cognitive 
diagrams  it  was  possible  to  compare  and  contrast  both  the  content  and  structure  of  the 
mental  representations  of  the  participants.  The  diagrams  were  a  useful  way  to  identify 
and  summarize  the  most  salient  issues,  actors  and  linkages  between  environmental  and 
political  problems  of  individuals  with  diversely  different  backgrounds  and  experiences.  I 
found  that  a  synthesis  of  the  issues  identified  in  the  summaries  was  a  useful  approach  for 
identifying  current  and  future  environmental  issues  that  might  lead  to  conflict  in  the 
Southern  African  region.  Disagreements  in  the  analyses  of  the  protocols  of  similar 
problems  highlighted  differences  of  opinion  in  how  specific  environmental  policy  issues 
should  be  framed  and  may  be  a  useful  indicator  of  fundamentally  different  policy  scripts 
or  ideologies  shared  by  relevant  political  elites  that  may  form  the  basis  of  future  political 
conflicts. 

Figures  2  and  3  illustrate  the  wide  variation  among  political  experts  who  were 
asked  to  think  aloud  about  the  same  general  forecasting  task,  to  wit,  what  current 
environmental  problems  are  likely  to  lead  to  future  political  problems?  Figure  2 
summarizes  the  verbal  protocol  of  a  female  African  American  who  was  a  career  State 
Department  Agricultural  Officer  working  in  the  US  embassy  in  an  African  country.  This 
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individual  initially  focused  on  the  region  and  stated  that  food  security  was  a  major 
problem  throughout  the  region.  The  analyst  quickly  shifted  to  listing  important 
agricultural  issues  in  South  Africa.  The  analyst  concluded  by  referencing  the  importance 
of  food  security  in  the  region.  This  sequence  is  interesting  given  the  extensive  amount  of 
past  research  documenting  that  political  solutions  are  typically  embedded  in  the  initial 
framing  of  the  problem  developed  by  political  decision  makers.  Perhaps  by  gaining  a 
better  understanding  of  the  preferred  framing  of  key  individuals  to  specific  problems 
within  a  given  organization,  we  may  better  understand  and  predict  future  choices  and 
policies  of  key  political  actors.  As  Cohen,  March,  and  Olsen  (1988:  296-7)  stress,  Aan 
organization  is  a  collection  of  choices  looking  for  problems Y(and)  solutions  looking  for 
issues  to  which  they  might  be  the  answer.  @ 


Figure  2 

EXAMPLES  OF  ALIST-LIKE0  PROBLEM  REPRESENTATIONS 


Ex  1:  Pure  List-like  framing 


Food  Security 

World  Food  Program  dead 

US  Dept,  of  Agriculture  not  sensitive  to  problems  of 
African  farmers  (South  Africa,  Botswana) 

South  Africa 

1- distribution  of  land 
who  owns, 

Who  will  pay, 

Different  organized  actors  by  sector, 

2-  water 

3- desertification 


(BATA) 


Other  issues  related  to  agriculture 

-Role  of  Banks  how  to  give  people  access  to  capital 

-Bioengineering  of  seeds 


-Few  are  focusing  on  broader  issue  of  food  security 
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Figure  3  summarizes  the  verbal  protocols  of  a  script-based  problem 
representation.  I  classified  this  representation  as  script-based  because  it  contains  several 
interrelated  themes  and  causal  inferences  that  can  be  used  to  explain  a  variety  of  current 
environmental  problems  in  the  region  and  within  a  specific  country,  Mozambique.  The 
partial  script-based  analyses  was  generated  by  a  former  US  military  officer  working  in  a 
senior  position  for  a  private  security  firm  in  Mozambique.  He  identified  and  related  three 
variable  clusters  -  environmental  degradation,  lack  of  state  capacity,  and  corruption  of 
political  officials  and  the  military  B  to  explain  both  general  conditions  found  in  countries 
throughout  the  region  and  to  explain  the  factors  fueling  specific  environmental  problems 
in  Mozambique  (See  Figure  3).  Thus,  environmental  degradation  throughout  Africa  and 
uncontrolled  fishing  off  the  coast  of  Mozambique  derive  from  extensive  poverty,  the 
limited  resources  of  the  state  to  cope  with  environmental  and  economic  problems,  and 
extensive  corruption  among  political  and  military  leaders  in  many  African  governments. 
Environmental  degradation  fueled  by  poverty,  the  lack  of  state  capacity,  and  corruption 
of  public  officials  are  identified  as  both  causes  and  consequences  of  environmental 
security  problems. 

Figure  3 

A  SCRIPT-BASED  APPROACH 

MACRO  THEME:  Mozambique  is  poor,  Aexpecter@,  Aungovernable@  state 

Threats  to  environment: 

# 1  THREAT  (All  African  states)  -ENVIRONMENTAL  DEGRADATION 

Population  density  &  Limited  resources  of  state...> 

...>  Inability  of  state  to  plan  or  enforce  regulations...>  Wide  scale  corruption 
among  senior  political  officials...> 

...>why  there  are  a  Alarge  number  of  basket  cases  in  Africa@ 

Environmental  degradation  manifested  as...> 

1  Deforestation 

-illegal  use  of  charcoal,  wood  ,  oil  Y>worsening  desertification  &  climate  change 
...>ANo  one:  looking  towards  future© 

EXAMPLE#1:  MOZAMBIQUE  Rural  women  walking  50  kilometers  for  fire  wood 
CONSTRAINTS  IN  MOZAMBIQUE: 

-No  agency  monitors  reforestation  programs 
-Widespread  crime 
-Defense  forces  in  disarray 

EXAMPLE#2:  Lake  Malawi  potential  as  protein  basin  for  region 
EXAMPLE#3:  2  ARape  of  Indian  Ocean© 
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Illegal  fishing  (Japanese  large  harvesters)  using  vacuum  methods  to 
or  Spaniards,  maybe  Russians)  ....>  Aclean  out  waters@...> 

...>depleting  fish  stocks  and  ...>destroying  coral 
CONSTRAINTS  &  EXAMPLES: 

US  organized  operation  to  cut  nets,  size  ships,  fines 
...>US  prohibition  against  countries  exports  w/o  protective  BUT 
turtle  nets  are  irrelevant  to  Mozambique=s  problems 

AND  MORE  IMPORTANTLY  Mozambique  government  doesn=t  recognize  there 
are  problems  with  environment  (forestry,  agriculture,  fishing)  OR  political  system 

Military  lack  capacity  or  resources  (Afor  a  parade@) . > 

soldiers  work  for  Atea  and  smokes  @ 

AND  ANobody  in  military  cares  @ 

#2  THREAT  ( ALL  COUNTRIES  IN  REGION)  IS  LACK  OF  CAPACITY 
...>little  joint  thinking  in  region 

Mozambique  has  2  boats  (100-150  ft.)  that  Abarely  float@ 

ANobody  cares  in  military© 

-No  Training  missions 

-No  Interest  in  effective  aid...>  No  Effect 

EXAMPLE:  US  tried  to  give  coast  guard  capability  (1995) 

...>Illegal  Fishing  continues  (Trawlers  with  12-20,000  ton  capacity)...> 

...>AND  Domestic  pilfering.. .> 

RESULT.. >only  4  tons  of  fish  coming  into  country  for  legal  sales... >Mozambique  is 
downloading  and  selling  off  capacity  with  cooperation  of  the  police 

#3  THREAT  CORRUPTION  < . competition  for  scare  resources  (in  poorest 

country  in  region) 

CONSTRAINTS: 

-Little  spending  on  social  welfare  (1.8%  healthcare;. 8  ed.) 

-Hollow  military  (11,000  or  3%  GNP  but  couldn=t  mobilize  Afor  a  parade©) 

-No  awareness  of  need  to  conserve  or  long-term  perspective 

Aif  can=t  eat  it,  sell  it,  use  it,  trade  it@ 

...>ANo  one  in  government  views  environment  as  a  Afront  burner  issue© 

Ministry  of  Defense  (military  in  name  only) 
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Armed  Forces  of  Senior  Army  General 
Mozambique 


army  police  custom  officials 
ANothing  moves  without  their  cooperations. .> 

..>must  pay  National  Directors  &  Minister  of  Defense  200/mo. 

...>No  recognition  or  government  pressure  for  social  welfare  programs 

2-  Solution:  1)  Pressure  from  outside  actors  (donor  countries, 

UN). .^environmental  planning  and  monitoring  & 

2)  Use  outside  contractors  for  public  functions  (e.g.,  customs  collection) 

This  analyst  concluded  with  a  simple  causal  map  that  identified  the  central  role  played  by 
the  military  in  Mozambique=s  current  political  system.  According  to  this  analyst,  this  military 
establishment  is  controlled  by  three  top  national  ministers  who  supplement  their  meager  official 
salaries  by  controlling  virtually  everything  coming  into  or  going  out  of  the  country.  The 
continuation  of  this  system  is  assured  by  the  fact  that  the  individuals  who  direct  the  army,  police, 
and  customs  are  key  players  in  this  political  system.  According  to  this  script,  solutions  to 
Mozambique=s  political  and  environmental  problems,  and  many  other  African  states,  will 
require  pressure  from  outside  donor  community  and  the  use  of  private  contractors  to  run  essential 
government  services . 

Despite  important  differences  in  the  content  and  structure  of  the  two  cognitive  diagrams, 
the  diagrams  suggest  that  forecasting  effort  about  environmental- security  linkages  in  the  region 
that  exclude  agricultural  issues  will  not  be  accepted  as  valid  or  relevant  to  Subject  1  (Figure  2). 

In  a  similar  vein,  forecasting  efforts  that  fail  to  incorporate  issues  related  to  poverty,  aid  efforts, 
and  government  corruption  in  Mozambique  and  throughout  Sub-saharan  Africa  are  unlikely  to  be 
accepted  as  valid  or  used  by  the  second  subject  (Figure  3). 

Conclusion 

The  basic  thesis  of  this  paper  is  that  forecasting  methods  and  forecasts  are  more  likely  to 
be  accepted  and  used,  if  the  results  are  related  to  the  implicit  model  of  the  problem  employed  by 
decision  makers  The  two-fold  classification  scheme  developed  is  offered  as  one  way  to 
determine  the  type  of  forecasting  methods  that  may  be  most  appropriate  for  a  particular 
forecasting  task.  When  the  problem  at  hand  is  an  ill- structured  problems,  past  studies  indicate  a 
need  for  an  analyst  to  understand  better  the  mental  representation  of  the  problem.  The  data  also 
support  the  proposition  that  political  experts  will  evidence  important  individual  differences  in 
the  way  they  represent  ill- structured  problems.  Thus,  the  mental  representations  of  specific 
problems  of  political  experts  may  be  expected  to  vary  in  structure  from  list-like  representations 
to  well-developed,  script-based  mental  representations.  Construction  of  summary  cognitive 
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diagrams  from  think  aloud  individual  protocols  or  analysis  of  small  group  discussions  about  the 
forecasting  problem  is  one  way  for  an  analyst  to  understand  better  the  implicit  model  of  the 
problem  employed  by  clients.  A  better  understanding  of  the  mental  representation(s)  of  end  users 
may  allow  the  forecaster  to  more  effectively  communicate  why  and  how  specific  forecasting 
methods  or  results  are  relevant  to  understanding  the  immediate  problem  at  hand. 
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Endnotes 


1.  Recent  events  in  Nigeria  illustrate  how  rapidly  economic  and  political  conditions  can  change 
in  less  developed  countries  that  are  dependent  on  a  single  primary  product  for  foreign  exchange 
earnings.  On  June  1,  2000  the  Nigerian  government  by  decree  raised  domestic  gasoline  prices  by 
50  percent.  This  sudden  price  increase  triggered  a  general  strike  that  quickly  spread  to  the  oil 
sector.  Less  than  a  week  after  the  onset  of  this  strike,  President  Obusegun  Obasanjo  apologized 
for  not  consulting  widely  on  sharp  increases  in  gasoline  prices.  Since  the  strike  quickly  crippled 
the  economy,  Obasanjo  was  desperate  to  end  the  strike.  Economic  turmoil  generated  by  a 
prolonged  strike  is  precisely  the  type  of  situation  that  triggered  military  coups  in  Nigeria  in  the 
past.  Thus,  Nigeria  in  a  few  weeks  moved  from  a  low-to-moderate  risk  of  a  military  coup  to  a 
high  coup-proneness. 

2. It  should  be  noted  that  professional  Aodds  makers©  including  expert  bridge  players,  gamblers 
and  National  Weather  Service  forecasters,  who  received  regular  feedback  about  the  accuracy  of 
specific  judgments,  did  not  exhibit  the  same  degree  of  resistance  to  changing  their  predictions  on 
the  basis  of  feedback  information  (Lichtenstein,  Lichoff,  and  Phillips,  1982;  Murphy  and 
Winkler,  1984). 

3.  While  care  must  be  taken  in  making  generalizations  from  a  non-random  sample,  interviewee 
were  knowledgeable  and  influential  individuals  who  had  extensive  experience  dealing  with 
political  and  environmental  problems  in  Southern  Africa.  Each  of  the  individual  who 
participated  in  this  study  had  past  experience  dealing  with  the  linkages  between  politics  and  at 
least  one  environmental  policy  area.  About  half  of  the  interviewees  were  government  officials. 
Their  occupations  ranged  from  cabinet  level  ministers,  heads  of  bureaucratic  departments  in 
national  or  provincial  levels  of  government,  and  officials  with  national  parks,  police,  or  military 
forces  who  had  recent  first-hand  experience  working  on  problems  relevant  to  the  concept  of 
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environmental  security.  Other  individuals  worked  in  senior  positions  for  international  or  regional 
organizations;  non-governmental  organizations  with  programs  in  the  region,  or  were  researchers 
working  in  universities,  policy  institutes,  or  non-governmental  organizations.  The  majority  of 
these  interviewees  worked  in  positions  that  focused  on  a  particular  set  of  environmental 
problems  (i.e.,  water,  forestry,  wildlife),  or  monitored  trends  about  a  particular  set  of  problems  in 
a  specific  country  (i.e.,  US  military  and  diplomatic  officials)  assigned  to  specific  countries  and 
policy  areas.  Approximately  20  per  cent  of  the  interviewees  worked  in  roles  that  required  them  to 
think  broadly  in  terms  of  the  linkages  between  environmental  and  security  policies  as  part  of 
their  current  job  requirement. 


20 


